Three-dimensional hyperspectral imaging systems and methods using a light detection and ranging (LIDAR) focal plane array

ABSTRACT

A system for three-dimensional hyperspectral imaging includes an illumination source configured to illuminate a target object; a dispersive element configured to spectrally separate light received from the target object into different colors; and a light detection and ranging focal plane array (FPA) configured to receive the light from the dispersive element, configured to acquire spatial information regarding the target object in one dimension in the plane of the FPA, configured to acquire spectral information in a second dimension in the plane of the FPA, wherein the second dimension is perpendicular to the first dimension, and configured to obtain information regarding the distance from the FPA to the target object by obtaining times of flight of at least two wavelengths, thereby imaging the target object in three dimensions and acquiring spectral information on at least one 3D point.

BACKGROUND

The invention relates generally to three-dimensional imaging systems andmethods and more particularly to three-dimensional hyperspectral imagingsystems and methods using a light detection and ranging (LIDAR) focalplane array.

SUMMARY

In one set of embodiments, a system for three-dimensional (3D)hyperspectral imaging includes an illumination source configured toilluminate a target object; a dispersive element configured tospectrally separate light received from the target object into differentcolors; and a light detection and ranging focal plane array (FPA)configured to receive the light from the dispersive element, configuredto acquire spatial information regarding the target object in onedimension in the plane of the FPA, configured to acquire spectralinformation in a second dimension in the plane of the FPA, wherein thesecond dimension is perpendicular to the first dimension, and configuredto obtain information regarding the distance from the FPA to the objectby obtaining times of flight of at least two wavelengths, therebyimaging the target object in three dimensions and acquiring spectralinformation on at least one 3D point.

In another set of embodiments, a system for three-dimensional (3D)hyperspectral imaging includes an illumination source configured toilluminate a target object using broadband laser light; an objectiveoptical element configured to receive light from the target object andto collect the light; a slit element configured to receive the lightfrom the objective optical element and to spatially filter the light; adispersive element configured to spectrally separate the light intodifferent colors; collimating optics configured to collimate the lightonto the dispersive element; focusing optics to refocus the collimatedlight; and a light detection and ranging focal plane array (FPA)configured to receive the refocused light from the focusing optics andto detect a single photon of light, configured to acquire spatialinformation regarding the target object in one dimension in the plane ofthe FPA, configured to acquire spectral information in a seconddimension in the plane of the FPA, wherein the second dimension isperpendicular to the first dimension, and configured to obtaininformation regarding the distance from the FPA to the object byobtaining times of flight of at least two wavelengths, thereby imagingthe target object in three dimensions and acquiring spectral informationon at least one 3D point, wherein at least two detections are performedto construct a histogram representing the spectral reflectivity of thetarget object.

In still another set of embodiments, a method for three-dimensional (3D)hyperspectral imaging includes the steps of: providing a 3Dhyperspectral imaging apparatus comprising an illumination source, adispersive element, and a light detection and ranging (LIDAR) focalplane array (FPA); illuminating a target object with the illuminationsource; using the dispersive element, spectrally separating the lightreceived from the target object into different colors; using the 3DLIDAR FPA, receiving the spectrally separated light, acquiring spatialinformation regarding the target object in one dimension in the plane ofthe FPA, and acquiring spectral information regarding the target objectin a second dimension in the plane of the FPA, wherein the seconddimension is perpendicular to the first dimension; and using the 3DLIDAR FPA, obtaining information regarding the distance from the FPA tothe object by obtaining times of flight of at least two wavelengths,thereby imaging the target object in three dimensions and acquiringspectral information on at least one 3D point.

In yet another set of embodiments, a method for three-dimensional (3D)hyperspectral imaging includes the steps of: providing a 3Dhyperspectral imaging apparatus comprising an illumination source, anobjective optical element, a slit element, collimating optics, adispersive element, focusing optics, and a light detection and ranging(LIDAR) focal plane array (FPA) configured to detect a single photon oflight; illuminating a target object with the illumination source;receiving and collecting light from the target object using theobjective optical element; using the slit element, spatially filteringthe light; using the collimating optics, collimating the light onto thedispersive element; using the dispersive element, spectrally separatingthe light into different colors; using the focusing optics, refocusingthe spectrally separated light; using the 3D LIDAR FPA, receiving therefocused light from the focusing optics; acquiring spatial informationregarding the target object in one dimension in the plane of the FPA,and acquiring spectral information regarding the target object in asecond dimension in the plane of the FPA, wherein the second dimensionis perpendicular to the first dimension; and using the 3D LIDAR FPA,obtaining information regarding the distance from the FPA to the targetobject by obtaining times of flight of at least two wavelengths, therebyimaging the target object in three dimensions and acquiring spectralinformation on at least one 3D point.

DESCRIPTION OF THE DRAWINGS

The accompanying drawings provide visual representations which will beused to more fully describe various representative embodiments and canbe used by those skilled in the art to better understand therepresentative embodiments disclosed herein and their advantages. Inthese drawings, like reference numerals identify corresponding elements.

FIG. 1 is a schematic illustration of a three-dimensional (3D)hyperspectral imaging system using a LIDAR focal plane array.

FIG. 2 is a drawing showing an overview of a 3D hyperspectral imagingsystem using a LIDAR focal plane array (FPA).

FIG. 3 is a schematic diagram of components of a 3D hyperspectralimaging system using a LIDAR focal plane array (FPA), showingcorresponding modeled data for a multilayer target object, wherein theFPA is configured to detect a single photon. FIG. 3A is a blown up viewof a portion of FIG. 3 depicting a pixel.

FIGS. 4A-4D are a set of intensity images acquired by a 3D hyperspectralimaging system using a LIDAR focal plane array, wherein the FPA isconfigured to detect multiple photons.

FIG. 4E is a schematic illustration of measurement of the range of thetarget (an “x-range-wavelength scatterplot”) using time-of-flightinformation by a 3D hyperspectral imaging system using a LIDAR focalplane array.

FIG. 4F is a histogram of range measurements for the data pointscomprised in FIGS. 4B-4D.

FIG. 5A shows a datacube generated by a passive hyperspectral imager.

FIG. 5B shows the view perpendicular to the x-y plane of FIG. 5A.

FIG. 5C shows a datacube generated by a 3D hyperspectral imaging systemusing a LIDAR focal plane array.

FIG. 6 is a flowchart of a method for 3D hyperspectral imaging using aLIDAR focal plane array.

FIG. 7 is a flowchart of a method for 3D hyperspectral imaging using aLIDAR focal plane array configured to detect a single photon of light.

DETAILED DESCRIPTION

While the present invention is susceptible of embodiment in manydifferent forms, there is shown in the drawings and will herein bedescribed in detail one or more specific embodiments, with theunderstanding that the present disclosure is to be considered asexemplary of the principles of the invention and not intended to limitthe invention to the specific embodiments shown and described. In thefollowing description and in the several figures of the drawings, likereference numerals are used to describe the same, similar orcorresponding parts in the several views of the drawings.

Multi-sensor data can be fused together to increase probability ofautomatic target recognition. Spectral 3D imaging light detection andranging (LIDAR) have been demonstrated using the multiple slit streaktube imaging LIDAR (STIL) for only a few wavelengths. In a monochromaticSTIL technique, a laser pulse source reformatted to a line illuminates atarget and the returning light pulse is received and directed toward aphotocathode with a slit. The received light pulse is converted toelectrons. Plates with ramping potential sweep the fan of electrons ontoa phosphor anode.

The time of return of the laser pulse is determined using the locationof the electrons on the phosphor anode. The light signal from the anodeis collected by a charge-coupled device (CCD) array. A STIL system,however, requires some prior knowledge of the location of an object.Moreover, permissible variations in range within a scene are limited bythe size of the CCD. Prior knowledge of the target range cannot beeasily gained using solely the STIL system. In addition, it isimpractical to create a 3D hyperspectral imaging system which willrequire too many slits for the STIL system.

Fused data sets derived from combining hyperspectral and LIDAR dataprovide opportunities to measure the structure of a target inthree-dimensions as well as classifying the materials that constitutethe object. These types of data sets have been used for forestinventory, global monitoring of deforestation events, and urbanstructure monitoring. Such fused data sets are commonly created fromhyperspectral and LIDAR data sets taken at different times. Accordingly,the possibility that objects may have moved in between the creation ofdata sets can impact the data's usefulness, especially for dynamic urbanscenes. In addition to such simultaneity issues, the hyperspectral andLIDAR sensors often have one or more of different array formats,different ground sample distances (GSD), and different perspectives onthe target of interest. These differences made the data fusion processand its automation non-trivial.

As a result, data processing often takes up a large amount of resourcesin the data product cycle and requires significant human intervention toresample the data and spatially overlapping them. Although designing andflying a combined instrument may reduce the complexity in producingfused data products, the hyperspectral imager—being a passiveinstrument—only collects data at times of day when the Sun is near itsnadir. Moreover, tall objects, such as buildings and trees, can castshadows, thereby creating holes in the hyperspectral images.

Embodiments of the invention provide a three-dimensional (3D)hyperspectral imaging system and method capable of providing threedimensional spatial information, a target's spectral signature on atleast one point, and temporal information on an emission induced by alaser pulse.

According to embodiments of the invention, a broadband pulsed lasersource may be spatially reformatted into a line illumination on a targetobject that is imaged across the spatial dimension on the FPA with mpicture elements or pixels. The spectral information from these m pixelsis recorded along the spectral dimension on the FPA, which has n pixels.At least one of the m by n pixels is capable of recording thetime-of-flight of the return laser pulse and multiple times-of-flightfor pixels that see multiple layers.

According to embodiments of the invention, temporal information on thelaser-pulse-induced emission may be observed by the time-of-flight (TOF)measurement of the return laser pulse. According to further embodimentsof the invention, the temporal information may make it possible tomeasure the lifetime of the excitation state of the target material.

According to embodiments of the invention, a target object may beilluminated by laser light. According to further embodiments of theinvention, objective optical element may receive the light returningfrom the object. The returning light may, according to embodiments ofthe invention, be focused through a split that spatially filters thereturning light into an array with m elements.

According to other embodiments of the invention, additional optics maycollimate the light onto a dispersive element to spectrally spread thelight into n different wavelength bins. According to still furtherembodiments of the invention, the spectrally dispersed light may then bedirected and refocused upon an m-by-n light detection and ranging(LIDAR) focal plane array (FPA).

According to embodiments of the invention, the system can be operated inpush-broom mode on an aircraft that scans the target.

According to further embodiments of the invention, at least one pixelcomprised in the 3D LIDAR FPA independently counts time to measure theTOF of the reflected laser pulse. The spatial profile of the lasersource is reformatted to a single line and matched to the field of viewin the spatial dimension of the FPA in order to optimize targetillumination. The laser source may comprise a wide spectral bandwidthlaser.

Embodiments of the invention provide an instrument that can map a targetin three dimensions. Embodiments of the invention provide an instrumentthat can measure the spectral information of a target. Embodiments ofthe invention provide an instrument that can map a target in threedimensions while simultaneously measuring its spectral information.

Embodiments of the invention can measure the excited state lifetime ofthe target material after excitation by a laser pulse.

Embodiments of the invention can measure a target hidden beneathcanopies and acquire 3D information regarding the object and spectralinformation for at least one 3D point. Thus, it is capable of 3Dhyperspectral foliage penetration by gating out the layers that areobscuring the intended target

According to embodiments of the invention, two systems and methods arepresented for using a LIDAR focal plane array.

According to embodiments of the invention, the system may operate at alow repetition rate with high pulse energy. According to theseembodiments, the FPA is configured to detect multiple photons.

According to embodiments of the invention, the FPA may be configured todetect a single photon.

According to embodiments of the invention, at least two detections maybe performed, and the at least two detections may be used to construct ahistogram representing the spectral reflectivity of the target object.

FIG. 1 is a schematic illustration of a 3D hyperspectral imager using aLIDAR focal plane array. In order to produce a three-dimensional imageof a target scene 110 as well as spectral information, a broadbandpulsed laser source 115 is spatially reformatted into a lineillumination across m pixels 120 on a target 110 that is imaged acrossthe spatial dimension on the FPA. The spectral information from these mpixels is recorded along the spectral dimension on the FPA, which has npixels 130. The n pixels 130 have different wavelengths corresponding totheir different positions along the vertical or spectral axis. At leastone of the m by n pixels is capable of recording the time-of-flight ofthe return laser pulse. Pixels that see multiple layers are capable ofrecording multiple times-of-flight.

The image width and length of this line illumination may be closelymatched to the field of view of the sensor for best performance.Although the laser beam profile can serve as a spatial filter, it ispreferable to use a slit to limit the amount of background noise. Anoptical element, such as a diffraction grating or a prism, disperses andseparates the return laser pulse 115 into different colors along thespectral direction of the focal plane array 140. A set of focusingoptics refocuses the collimated light onto the focal plane array. Havingthe light dispersed by a grating or a prism and being on different partsof the target 110 along the x axis, the light pulses returned from thestep target that are collected by the focusing optics will land ondifferent parts of the focal plane array 140 along the spatial andspectral directions and at different times. By its ability to measuretime-of-flight of a return pulse for each picture element on the LIDARfocal plane array 140, the system acquires spatial information along thex axis 150 and the z axis 160 along with spectral information for atleast one picture element through at least one laser flash. Uponscanning the system along the y axis 170 in a push broom operationalmode, spatial information of the target 110 for three spatial dimensionscan be gathered as well as spectral information for each pictureelement. The spectral bandwidth can be determined by the array size andthe dispersive element. Accordingly, a scene 110 can be imaged in threedimensions, x, y, and z. Spectral information is gathered for these 3Ddata points.

Imaging systems that use a LIDAR FPA can perform 3D imaging of an entirescene in a single laser flash. For applications where the target ismoving at a very high speed, such systems essentially freeze the objectin its trajectory at the instant of a single laser flash.

Instead of using the LIDAR camera to image an entire scene in a singlelaser flash, according to embodiments of the invention, a slit and adispersive element can be placed in the optical path to convert thecamera into a 3D hyperspectral imager.

The spatial profile of the laser source must be reformatted to a singleline and matched to the field of view in the spatial dimension of theFPA so that the illumination of the target is optimized. To measure thespectral reflectance signature of a target and to map the target inthree dimensions, the laser must have a wide enough spectral band tocover the spectral region of interest. For example, a wide spectralsource can be used to measure the health of trees since dry leaves havemarkedly different spectral reflectance in the near infrared while atthe same time map out the leave density in 3D. In addition, embodimentsof the invention may be used in pump-probe techniques that can increasespecificity and provide higher accuracy in chemical identification thatis often obscured by spectral interference from surrounding chemicalspecies.

FIG. 2 is a drawing showing an overview of a 3D hyperspectral imagingsystem to image a target 110 using a LIDAR focal plane array (FPA) 140.

The illumination source (not shown) can comprise a laser source with awide spectral bandwidth. For example, a broadband laser source can beused with a pulse width on the order of a few nanoseconds. To measurethe spectral reflectance signature of a target 110 and to map the target110 in three dimensions, the illumination source must have a wide enoughspectral band to cover the spectral region of interest. For example, theillumination source may be a commercial off-the-shelf (COTS) broadbandKoheras SuperK Compact Laser from NKT Photonics (formerly Koheras NS) ofCopenhagen, Denmark. This laser system, like most fiber lasers, operatesat a high repetition rate with low pulse energy. Thus, according toembodiments of the invention, a single-photon sensitive LIDAR FPA withhigh frame rate can be used.

According to embodiments of the invention, the target object 110 isilluminated by a line of illumination. The spatial profile of theillumination source is reformatted to a single line and matched to thefield of view in the spatial direction of the FPA 140 so that theillumination of the target 110 is optimized. The LIDAR focal plane array140 samples the return signal after reflection by the target 110 at asampling time interval of Δt.

As a simplified illustration, the target 110 in the schematic diagram iscomprised of three step surfaces 210A, 210B, and 210C with reflectanceat three different wavelengths of light to reflect a return pulse 215,comprising three component return pulses 215A, 215B, and 215C withcorresponding paths of different lengths. The surface 210A that reflectsλ₁ has the longest path 215A to the FPA 140, the surface 210B thatreflects λ₂ has a path 215B of intermediate length to the FPA 140 andthe surface 210C that reflects λ₃ has the shortest path 215C to the FPA140. Component return pulse 215C reflected from the surface 210C thatreflects λ₃ is detected at an earlier time t₁, component return pulse215B from the surface 210B that reflects λ₂ is detected later at anintermediate time t_(z) and component return pulse 215A from the surface210A that reflects λ₁ is detected at a later time t₃.

The system 200 further comprises objective optical element 230 and aspectrometer 235. The spectrometer 235 comprises a spectrometer entrance240, collimating optics 250, a dispersive element 260, and focusingoptics 270.

The objective optical element 230 collects and focuses the line ofillumination through the spectrometer entrance 240. The spectrometerentrance 240 may be a slit 240. The spectrometer entrance 240 spatiallyfilters the line of illumination and defines a spatial line in thespatial dimension of the focal plane array 140. The collimating optics250 collimates the light onto the dispersive element 260. The dispersiveelement 260 spreads the line of light in the spectral direction. Forexample, the dispersive element 260 may be one of a grating 260 and aprism 260. The focusing optics 270 then focuses the light onto the FPA140. The spectrally dispersed light is incident on the m-by-n 3D LIDARfocal plane array 140.

The FPA 140 comprises an m-by-n array with pixels 280A, 280B, 280C, andso on that measures the profile of the component return pulses 215A-215Cusing time slices 290A-290C. The reflected component return pulses 215A,215B, and 215C have different times-of-flight and are respectivelydetected in different time bins 290A, 290B, and 290C on the LIDAR focalplane array 140. Generally there will be an approximate fixed samplingtime interval Δt 295 between successive samples. The sampling intervalΔt 295 is usually on the order of a nanosecond. The time differencebetween the time t₁ at which time bin 290C is sampled and the time t₂ atwhich time bin 290B is sampled may be a large multiple of Δt. Similarly,the time difference between the time t₂ at which time bin 290B issampled and the time t₃ at which time bin 290A is sampled may be a largemultiple of Δt.

At least one of the pixels 280A, 280B, 280C and so on can be set totrigger independently, allowing target objects at vastly differentranges to be imaged in a single laser flash. Upon triggering, each pixel280A, 280B, 280C uses its buffer to record a profile of the return pulse215. In addition, each pixel 280A, 280B, 280C records the time of thetriggering event, also known as the course time-of-flight (TOF) of thereturn pulse 215. The recorded profile of the return pulse 215 can helpto determine the time-of-flight more accurately through curve fitting,centroiding, or other statistical methods. Accordingly, this techniqueavoids the limitation of the range resolution by the sampling intervalthat would otherwise occur. By using the measured course time-of-flightand the peak position of the return pulse profile from the time samples,the system can determine the locations of objects within a scene thatmay be very far apart from each other.

The readout integrated circuit (ROIC) (not shown) can be operated in twodifferent modes, a threshold mode or a gated mode. In the thresholdmode, each pixel on the ROIC starts to count time at the moment when theillumination source (not shown) fires a pulse. As soon as the returnpulse 215 crosses a preset threshold on a particular pixel 280A, 280B,280C, and so on, that pixel will sample the return pulse waveform andstops the clock after a ring buffer limit is reached. Since each pixel280A, 280B, 280C, and so on triggers independently in the thresholdmode, the system 200 has a large dynamic range in measuring widelyvarying TOF in a single scene.

In contrast, in the gated mode, the FPA 140 collects informationregarding a single frame at a preset time. The gated mode requires someprevious knowledge of the target range, but it is useful insystematically mapping the shape of a target object 110. The gated modeis also useful to determine one or more of the vertical atmosphericprofile and the lifetime of an excited state for a target 110 that isrelatively time invariant during data collection. Operating in the gatedmode provides the opportunity to measure the fluorescence lifetime of atarget 110. For fluorescence measurements, a single wavelength pulselaser is used to excite the target material and the system 200 thenmeasures the return spectra as a function of time with a time samplinginterval of Δt 295. Knowledge of the target range can be gained byfiring a single laser shot with the ROIC (not shown) operating inthreshold mode and then reverting to the gated mode to accomplish themeasurements.

The system 200 can be operated in a push-broom mode, for example,push-broom mode on an aircraft that scans the target. Whiskbroom mode isalso possible, in which case an additional scanning mechanism is used.

The system 200 can also be mounted on top of a vehicle and configured toscan in 360 degrees so as to map 3D points and so as to acquire spatialand spectral information for 3D points surrounding the vehicle. Suchembodiments may prove useful for situation awareness applications.

The system 200 was verified experimentally as able to measure spectralreflectance of a target and to determine its location in 3D space at asmall spot. The time-of-flight of the return signal and the inherentspectrum of the broadband laser source were both simultaneously measuredby the system 200.

FIG. 3 is a schematic diagram of components of a 3D hyperspectralimaging system using a LIDAR focal plane array (FPA) showingcorresponding modeled data for a multilayer target object, wherein theFPA is configured to detect a single photon. FIG. 3A is a blown up viewof a portion of FIG. 3 depicting a pixel.

The active 3D HSI system 300 uses a single-photon sensitive LIDAR focalplane array (FPA) 140 that operates in linear mode and at a frame rateon the order of tens of kilohertz. For long range applications, thenumber of photons returned per laser shot may be low. Accordingly, thespectrum of a ground pixel may be built up through many laser shots.This is feasible since both the laser 310 and the LIDAR FPA 140 canoperate at a high repetition rate on the order of tens of kilohertz.

A trigger timer 320 synchronizes the laser source 310 and the LIDAR FPA140. The laser source 310 will emit a spectral waveform 325 exhibiting atemporal pulse width 330 on the order of a nanosecond as indicated bythe temporal waveform 335. The laser beam is then spatially reformattedby a beam reformatter 340 into a line illumination. The reformatted beamthen propagates to the target object 110, whose pixel 342 comprisesmultiple layers, for example, layers 344A and 344B. The target object110 may, for example, be a target object 110 comprising twosubstantially flat layers as in the example shown in FIG. 3 forillustration purposes. Real targets may have more complex layering. Toplayer spectral reflectance waveform 345 and bottom layer spectralreflectance waveform 350 respectively indicate the spectral reflectanceof the top layer and of the bottom layer of target object 110. The toplayer spectral reflectance waveform 345 is a representative spectralreflectance waveform for healthy plant leaves, while the bottom layerspectral reflectance waveform 350 is a representative spectralreflectance waveform for dry plant leaves. The bottom layer spectralreflectance waveform 350 is relatively featureless compared to that ofthe top layer spectral reflectance waveform 345.

To simplify matters without a loss of generality, the followingdiscussion will focus on the signal coming from a pixel 342 comprised inthe target object 110, where the pixel 342 covers two layers 344A-344B.The signal from the pixel in target object 110 is detected by a columnof device pixels comprised in the LIDAR FPA 140. As indicated byreceived spectral reflectance waveforms 215, the spectral return pulsewill typically be approximately equal to the product of the spectrum ofthe laser 310 and the spectral reflectance of the target object 110 andthe atmospheric transmission (not shown). As seen by the single pixel342, the top layer of the target object 110 is closer to the FPA 140 andtherefore typically returns a return signal at an earlier time t₁ 380Awith a spectral profile as indicated by first received spectralreflectance waveform 355A. As seen by the single pixel 342, the bottomlayer of the target object 110 is farther from the FPA 140 and thereforetypically returns a spectral signal at a later time t₂ 380B with aspectral profile as indicated by second received spectral reflectancewaveform 355B.

These return signals 215A-215C are collected by the objective opticalelement 230. Return signals 215A-215C comprise light pulses withtemporal profiles approximately as indicated by the temporal waveform335 for significantly thin layers. The imaging spectrometer 235disperses the light across the LIDAR FPA 140 to separate out thespectral components and thereby separating a single pulse of light intomany pulses of light with different center wavelengths along thespectral dimension of the FPA. For example, the spectrometer maycomprise a COTS imaging spectrometer from HORIBA Instruments, Inc. ofSanta Clara, Calif.

A column of detectors collects the return signal 215 for the singlepixel 342. Pixels 280A, 280B, 280C, and so on comprised in the FPA 140are synchronized by the trigger timer 320. The pixels 280A, 280B, 280C,and so on sample the return signal 215 within an acquisition window at aparticular center wavelength. The trigger timer 320 issues one or moresynchronization pulses 360A and 360B. To reduce the amount ofunnecessary data and to prevent saturation of the detectors at the timeof the laser shot, a time delay 365 is introduced between the triggerpulse 360A and the start time 370 when data acquisition begins. Theduration of the data acquisition time window 375 during which data maybe acquired is selected based on the expected distance between theclosest and farthest objects 110 to be imaged.

Assuming that the target object 110 as seen by the pixel 342 comprises aplurality of layers 344A-344B, for example, a top layer 344A and abottom layer 344B. The return signals 215 from the two layers will berespectively detected at two nearby times, at a first time 380A (t₁) forthe top layer 344A and at a second time 380B (t₂) for the bottom layer344B. More complex structures would cause the return signals 215 tospread in time with temporal centers close to the first time 380A forthe top layer 344A and close to the second time 380B for the bottomlayer 344B. Waveforms 355A and 355B respectively carry informationregarding the top layer spectral reflectance waveform 345 and the bottomlayer spectral reflectance waveform 350. The waveforms 355A and 355B arealtered by the spectral waveform 325 and by atmospheric transmission.

Upon analysis of the atmospheric conditions and using the spectralwaveform 325, the spectral reflectance of the top layer 344A and of thebottom layer 344B, respectively as indicated by the top and bottomspectral reflectance waveforms 345 and 350, may be derived. Firstwaveforms 385A and 385B respectively represent histograms built up aftera few laser shots for the top layer 344A and for the bottom layer 344B.Second waveforms 390A and 390B respectively represent histograms builtup after half of the total number N of shots for the top layer 344A andfor the bottom layer 344B. Third waveforms 395A and 395B respectivelyrepresent histograms built up after N shots for the top layer 344A andfor the bottom layer 344B. While the preceding paragraphs of thisexample are applicable to embodiments in which multiple photons aredetected, exemplary waveforms 385A, 385B, 390A, 390B, 395A, and 395Bcorrespond to embodiments in which the FPA detects a single photon.

The spectrum is built up as a histogram as can be seen by reviewing thesix waveforms 385A-385B, 390A-390B, and 395A-395B. By measuring thespectral return as a function of time for each ground pixel with thesensor moving and operating in a push-broom fashion, a three-dimensional(3D) image is acquired with spectral information for at least one 3Dpoint.

FIGS. 4A-4D are a set of intensity images acquired by a 3D hyperspectralimaging system using a 3D LIDAR focal plane array, wherein the imagingsystem is configured to detect multiple photons. The target comprises agraded spectralon panel having substantially uniform spectralreflectivity in the near infrared region, but with regions of graduallydecreasing overall reflectivity. The images depict the brightest part ofthe panel and were obtained using as an illumination source 310 amonochromatic Big Sky laser manufactured by Quantel Technologies ofFrance with a diffuser at its exit port and a broadband infrared lasersource that has a spectral profile that spans across the near infraredregion. The imaging spectrometer has a dispersive element that can beset to different angles to look at different spectral regions. Dataacquisition using the Big Sky laser with a diffuser and using thebroadband source as illumination source were not performedsimultaneously. The images were taken using a single-pulse samplingflash LIDAR camera manufactured by Advanced Scientific Concepts (ASC) ofSanta Barbara, Calif. with its long pass filter removed. Images takenusing the Big Sky laser with a diffuser form a narrow line that extendsacross the spatial x direction as shown in FIG. 4A and occurs at awavelength of approximately 1574 nm.

Since the spot size of the broadband illuminator covers approximatelyten pixels, it reveals itself as a relatively thick line that extendsalong the spectral direction for different grating positions as shown inFIGS. 4B-4D. These data sets were taken with the dispersive elementcomprised in the imaging spectrometer set at different angles. Thebroadband source was not reformatted as a line due to lack ofsensitivity of this FPA. A single-photon sensitive FPA would provide thenecessary parameter to allow full demonstration of the currentinvention.

FIG. 4E is a schematic illustration of measurement of the range of thetarget (an “x-range-wavelength scatterplot”) using time-of-flightinformation by a 3D hyperspectral imaging system using a 3D LIDAR focalplane array. The intensity images depicted in FIGS. 4A-4D appear in thisfigure with digital counts indicated by the scale 420. The dotted line430 shows the projection onto the wavelength axis of the intensity image450 depicted in FIG. 4A, clarifying that the wavelength for thisparticular example is approximately 1574 nm. Since the target comprisesa spectralon panel with uniform spectral reflectivity in the nearinfrared region, the variation in intensity along the spectral directionrepresents the spectrum of the broadband laser source. Using thetime-of-flight information, embodiments of the invention can measure therange of the target. FIG. 4E plots the location of the target 110 inthousandths of feet along the x-axis, the range of the target 110 infeet along the z-axis, and along the vertical axis, the spectralreflectivity of the target 110 as a function of wavelength innanometers. For a field system, the field of view and distance to thetarget could be expanded more than is indicated in this example. Inaddition, a dispersive element with lower dispersion may be used tocover a wider spectral region and improve signal-to-noise ratio.

FIG. 4F is a histogram of range measurements for the data pointscomprised in FIGS. 4B-4D. The measured mean distance between theinstrument and the flat spectralon panel was approximately 50 feet witha standard deviation of approximately 0.57 feet. The spread in thehistogram is principally caused by detection statistics and residualsafter correcting for a correlation between range and intensity. Thiscorrelation between range and intensity is an inherent problem with theparticular LIDAR FPA used and is not inherent in the 3D hyperspectralimaging systems and methods according to embodiments of this invention.The distribution of range measurements for the data points comprised inFIG. 4A (not shown) has a slightly smaller standard deviation. Itsspread is about 0.51 feet and is a result of a more uniform illuminationproduced by the diffuser at the laser output port. Range measurementsfor data points comprised in FIG. 4A are not included in FIG. 4F so thatthese data points will not skew the distribution of range measurementsgathered by using the broadband laser source. The similar rangeresolutions for the data points acquired using these two laser sourcesdemonstrate that the range-intensity correlation is independent ofwavelength in the spectral region comprising the data points illustratedin FIG. 4B-4D. These results show that 3D hyperspectral imaging using aLIDAR FPA according to embodiments of the invention is highly feasible.

According to embodiments of the invention, using an m-by-n 3D LIDARfocal plane array, a dispersive optical element at the fore-optics ofthe imager, and a pulsed laser source, a 3D hyperspectral imaging systemcan provide a data product with three spatial dimensions, andhyperspectral information for acquired 3D points. These data productdimensions may also consist of spatial information, the spectralsignature of a target, and temporal information on the emission inducedby a laser pulse. The temporal information either provides atime-of-flight (TOF) measurement of the return laser pulse, which cansimply be converted to the third spatial dimension, or a measure of theexcitation state lifetime of the target material after excitation by alaser pulse.

FIGS. 5A-5C illustrate the advantages of the current invention over apassive hyperspectral imager or a combined hyperspectral/LIDAR system.FIG. 5A shows a datacube generated by a passive hyperspectral imager.FIG. 5B shows the view perpendicular to the x-y plane of FIG. 5A.

As illustrated in FIG. 5A, a passive hyperspectral imager acquires adata cube 505 comprising a spatial component in the x direction, aspatial component in the y direction, and the spectral component in thez direction. A passive hyperspectral imager is unable to measure thez-spatial component of a scene. The true dimensions of a target object110 are often lost in such 2D images as a result of the camera's viewingfrustum and look angle. For example, aerial images of a tall buildingoften make the top of a building seem wider than the bottom of thebuilding as a result of the viewing frustum. From certain angles ofobservation, an object 110 may even appear distorted.

Data registration using a combined hyperspectral/LIDAR system cancorrect for such an effect. Such a combined system would comprise asingle wavelength LIDAR system and a passive hyperspectral imager thatcould have a resolution that is markedly different from that of thesingle wavelength LIDAR system. Such a combined instrument may be ableto correct for spatial distortions but will suffer from otherdeleterious effects such as shadowing effects. Shadowing effects cause aloss of spectral information as illustrated in FIG. 5B. As shown in FIG.5B, a target 510 partially lies in the shadow 515 cast by two nearbytrees 520A and 520B. When this occurs, the final data product of apassive hyperspectral imager or a combined hyperspectral/LIDAR systemwill have holes in the data cube 505—particularly in the spectraldirection of the data cube 505. Also, the time of data collection forthese types of instruments is often restricted to daytime betweenapproximately 10:00 AM and 2:00 PM, when the solar zenith angle is closeto normal. Such a combined hyperspectral/LIDAR system cannot performhyperspectral foliage penetration, and currently no system on the markethas such a capability

FIG. 5C shows a datacube 530 generated by a 3D hyperspectral imager 200using a LIDAR focal plane array 140. An active 3D hyperspectral imager200, such as the one described by this invention can minimize or eveneliminate most of the aforementioned problems of a passive hyperspectralimager or a combined hyperspectral/LIDAR system. Datacube 530 shows thetype of data that are collected by an active 3D hyperspectral imagerwhose data components comprise spatial-spatial-spatial-spectralcomponents. The terrain of the target scene 110 can now be mapped witheach pixel element comprising information regarding three spatialdimensional components and the spectral reflectance information. Thetarget 510 that was shadowed in FIGS. 5A and 5B by nearby trees 520A and520B is now fully illuminated by the white light laser source accordingto embodiments of the invention. As a result, the number of holes in thedata cube 530 is minimized.

A target object 110 that may be hidden underneath a canopy may also bedetected using the current invention where gaps that occur naturally incanopies may be exploited by the white light laser source—providing aunique opportunity to perform hyperspectral foliage penetration.Morphological analysis of an object and of a scene can now be exploited.For example, these techniques may be used to more precisely measure thedensity and health of a forest for carbon sequestration measurementswhile at the same time improving the certainty of target identificationeven when the target is hidden beneath a canopy. Finally, datacollection using such a system will not be restricted to any particulartime of day and will be able to acquire high quality data both night andday.

FIG. 6 is a flowchart of a method 600 for three-dimensional (3D)hyperspectral imaging using a LIDAR focal plane array. The order of thesteps in the method 600 is not constrained to that shown in FIG. 6 ordescribed in the following discussion. Several of the steps could occurin a different order without affecting the final result.

In block 610, a three-dimensional (3D) hyperspectral imaging apparatuscomprising an illumination source, a dispersive element, and a lightdetection and ranging (LIDAR) focal plane array (FPA) is provided. Block610 then transfers control to block 620.

In block 620, a target object is illuminated with the illuminationsource. Block 620 then transfers control to block 630.

In block 630, using the dispersive element, light received from thetarget object is spectrally separated into different colors. Block 630then transfers control to block 640.

In block 640, using the 3D LIDAR FPA, spatial information is acquiredregarding the target object in one dimension in the plane of the FPA andspectral information is acquired regarding the target object in a seconddimension in the plane of the FPA, wherein the second dimension isperpendicular to the first dimension. Block 640 then transfers controlto block 650.

In block 650, using the 3D LIDAR FPA, information is obtained regardingthe distance from the FPA to the object by obtaining times of flight ofat least two wavelengths, thereby imaging the target object in threedimensions and acquiring spectral information on at least one 3D point.Block 650 then terminates the process.

FIG. 7 is a flowchart of a method 700 for three-dimensional (3D)hyperspectral imaging using a LIDAR focal plane array. The order of thesteps in the method 700 is not constrained to that shown in FIG. 7 ordescribed in the following discussion. Several of the steps could occurin a different order without affecting the final result.

In block 710, a three-dimensional (3D) hyperspectral imaging apparatuscomprising an illumination source, an objective optical element, a slitelement, collimating optics, a dispersive element, and a light detectionand ranging (LIDAR) focal plane array (FPA) configured to detect asingle photon of light is provided. Block 710 then transfers control toblock 720.

In block 720, a target object is illuminated with the illuminationsource. Block 720 then transfers control to block 730.

In block 730, light is received and collected from the object using theobjective optical element. Block 730 then transfers control to block740.

In block 740, the light is spatially filtered using the slit element.Block 740 then transfers control to block 750.

In block 750, the light is collimated onto the dispersive element usingthe collimating optics. Block 750 then transfers control to block 760.

In block 760, using the dispersive element, the light is spectrallyseparated into different colors. Block 760 then transfers control toblock 770.

In block 770, using the focusing optics, the spectrally separated lightis refocused. Block 770 then transfers control to block 775.

In block 775, using the 3D LIDAR FPA, the refocused light is receivedfrom the focusing optics, spatial information is acquired regarding thetarget object in one dimension in the plane of the FPA and spectralinformation is acquired regarding the target object in a seconddimension in the plane of the FPA, wherein the second dimension isperpendicular to the first dimension. Block 770 then transfers controlto block 780.

In block 780, using the 3D LIDAR FPA, information is obtained regardingthe distance from the FPA to the object by obtaining times of flight ofat least two wavelengths, thereby imaging the target object in threedimensions and acquiring spectral information on at least one 3D point.Block 780 then terminates the process.

The Geiger-mode camera has a maximum frame rate that almost matches therepetition rate of the white light source and is single photon sensitivewithout any range-intensity effect. The biggest drawback of using theGeiger-mode camera is the non-proportional gain of the detectors. TheGeiger-mode camera operates by detecting a single event per laser pulse.Events are recorded in time without any amplitude information of thereturn signal. Thus the gain for the detectors is not linear. To producea hyperspectral image, it is necessary to take multiple laser flashes ofdata to build up a histogram to determine the spectral reflectivity of atarget. Intensity information gather by a Geiger-mode detector iscomplicated by its non-linearity. A much better detector to use is theintensified photodiode that has single-photon sensitivity andproportional gain and high dynamic range in photon counts/second.

By replacing with a single device a suite of instruments required togather both hyperspectral and 3D images, embodiments of the inventionmay reduce the number of instruments flown on environmental monitoringmissions. Accordingly, embodiments of the invention may reduce the size,weight, and power of the required instrumentation by replacing the suiteof instruments with a single device capable of producing even morecomprehensive data products.

There are several advantages that can apply to using a 3D LIDAR FPA toconstruct a hyperspectral 3D LIDAR system. When compared to a suite ofinstruments to gather both hyperspectral and 3D images, the active 3Dhyperspectral imager will reduce the number of instruments thus reducingthe size, weight, and power by replacing the suite of instrument with asingle instrument. In addition, using a single instrument thatsimultaneously collect different types of data product would producebetter coordinated results and reduce post-processing efforts.

A further advantage can be that an active 3D hyperspectral imagerremoves the complications of data fusion, spectrally and spatiallyresolving a target in three dimensions, and providing fast turn-arounddata sets that have absolute simultaneity and exact perspective of theobjects for both types of data.

Another advantage can be that according to embodiments of the invention,high spatial coverage may be achieved with simultaneous detection ofover 100 different wavelengths.

Additionally, a still further advantage is that it may be possible toperform chemical analysis of the return signal. The results can then beused to map a chemical plume of the target object.

Embodiments of the invention can measure the spectral reflectance of atarget on collected 3D points.

Embodiments of the invention may probe objects hidden beneath canopies.Embodiments of the invention may measure the spectral reflectance of anobject hidden, for example, beneath a tree, thereby providing a methodto perform hyperspectral foliage penetration.

Embodiments of the invention that simultaneously collect different typesof data may produce better coordinated results and may reduce the timedelays, expense, and trouble associated with the requiredpost-processing time.

While the above representative embodiments have been described withcertain components in exemplary configurations, it will be understood byone of ordinary skill in the art that other representative embodimentscan be implemented using different configurations and/or differentcomponents. For example, it will be understood by one of ordinary skillin the art that the order of certain fabrication steps and certaincomponents can be altered without substantially impairing thefunctioning of the invention.

The representative embodiments and disclosed subject matter, which havebeen described in detail herein, have been presented by way of exampleand illustration and not by way of limitation. It will be understood bythose skilled in the art that various changes may be made in the formand details of the described embodiments resulting in equivalentembodiments that remain within the scope of the appended claims. It isintended, therefore, that the subject matter in the above descriptionshall be interpreted as illustrative and shall not be interpreted in alimiting sense. The invention is defined by the following claims.

I claim:
 1. A system for three-dimensional (3D) hyperspectral imaging,comprising: an illumination source configured to illuminate a targetobject; a dispersive element configured to spectrally separate lightreceived from the target object into different colors; and a lightdetection and ranging (LIDAR) focal plane array (FPA) configured toreceive the light from the dispersive element, configured to acquirespatial information regarding the target object in one dimension in theplane of the FPA, the FPA comprising an integrated circuit configured tooperate in a gated mode, the FPA configured to acquire spectralinformation in a second dimension in the plane of the FPA, wherein thesecond dimension is perpendicular to the first dimension, configured toobtain information regarding a distance from the FPA to the targetobject by obtaining different times of flight of at least twowavelengths, and wherein the FPA is configured to measure a fluorescencelifetime of the target object, thereby hyperspectrally imaging thetarget object in three dimensions and acquiring spectral information onat least one 3D point.
 2. The system of claim 1, further comprising anobjective optical element, the objective optical element beingconfigured to receive light from the target object and to collect thelight.
 3. The system of claim 1, further comprising a slit elementconfigured to spatially filter the light.
 4. The system of claim 3,wherein the slit element is further configured to spatially filter thelight so as to define a spatial line in the spatial dimension of theFPA.
 5. The system of claim 1, further comprising collimating opticsconfigured to collimate the light.
 6. The system of claim 1, furthercomprising focusing optics configured to refocus the light onto the FPA.7. The system of claim 1, wherein the FPA samples the return signal at aselected sampling time interval.
 8. The system of claim 1, wherein thesystem operates in push-broom mode.
 9. The system of claim 1, whereinthe system operates in whisk-broom mode.
 10. The system of claim 1,wherein the illumination source comprises a white light laser.
 11. Thesystem of claim 1, wherein the illumination source is spatiallyreformatted into a line.
 12. The system of claim 11, wherein the line isclosely matched to the field of view of the FPA so as to optimizeperformance.
 13. The system of claim 1, wherein the system is configuredto measure spectral information regarding the target object.
 14. Thesystem of claim 1, wherein the system is configured to measure anexcited state lifetime of the target object following illumination. 15.The system of claim 1, wherein the system is configured to measurespectral information regarding the target object hidden beneath acanopy.
 16. The system of claim 1, wherein the system is configured toremove shadowing effects and to reduce data holes in a hyperspectraldata cube.
 17. The system of claim 1, wherein existing knowledge of thetarget range is used by the integrated circuit to determine the timewhen counting time begins for at least one pixel comprised in the FPA.18. The system of claim 1, wherein at least two detections areperformed, and wherein the at least two detections are used to constructa histogram representing the spectral reflectivity of the target object.19. A method for three-dimensional (3D) hyperspectral imaging,comprising: providing a three-dimensional (3D) hyperspectral imagingapparatus comprising an illumination source, a dispersive element, and alight detection and ranging (LIDAR) focal plane array (FPA), the FPAcomprising an integrated circuit configured to operate in a gated mode;illuminating a target object with the illumination source; using thedispersive element, spectrally separating light received from the targetobject into different colors; using the 3D LIDAR FPA, receiving thespectrally separated light, acquiring spatial information regarding thetarget object in one dimension in the plane of the FPA, measuring returnspectra as a function of time, measuring a fluorescence lifetime of thetarget object, and acquiring spectral information regarding the targetobject in a second dimension in the plane of the FPA, wherein the seconddimension is perpendicular to the first dimension; and using the 3DLIDAR FPA, obtaining information regarding the distance from the FPA tothe target object by obtaining different times of flight of at least twowavelengths, thereby hyperspectrally imaging the target object in threedimensions and acquiring spectral information on at least one 3D point.20. A system for three-dimensional (3D) hyperspectral imaging,comprising: an illumination source configured to illuminate a targetobject; a dispersive element configured to spectrally separate lightreceived from the target object into different colors; and a focal planearray (FPA) configured to receive the light from the dispersive element,the FPA comprising an integrated circuit configured to operate in agated mode, configured to acquire spatial information regarding thetarget object in one dimension in the plane of the FPA, configured toacquire spectral information in a second dimension in the plane of theFPA, wherein the second dimension is perpendicular to the firstdimension, configured to obtain information regarding the distance fromthe FPA to the target object by obtaining different times of flight ofat least two wavelengths, and wherein the FPA is configured to measure afluorescence lifetime of the target object, thereby acquiringhyperspectral information on at least one 3D point.
 21. A system forthree-dimensional (3D) hyperspectral imaging, comprising: anillumination source configured to illuminate a target object; adispersive element configured to spectrally separate light received fromthe target object into different colors; and a focal plane array (FPA)configured to receive the light from the dispersive element, the FPAcomprising an integrated circuit configured to operate in a gated mode,configured to acquire spatial information regarding the target object inone dimension in the plane of the FPA, configured to acquire spectralinformation in a second dimension in the plane of the FPA, wherein thesecond dimension is perpendicular to the first dimension, configured toobtain information regarding the distance from the FPA to the targetobject by obtaining different times of flight of at least twowavelengths, and wherein the FPA is configured to measure a fluorescencelifetime of the target object, thereby acquiring hyperspectralinformation on at least one 3D point, wherein the target object is amultilayer target object, and wherein at least two of the differenttimes of flight are recordable by a single pixel that sees more than onelayer of the multilayer target object.